CIM-Ollama

Get up and running with large language models locally.

Currently supported Models.

A few selected models we tend to favor...

The image size is currently 32G, so about 7 or 8 models can be cached. You are free to change the size of the drive in configuration.nix

Model Parameters Size Download
Code Llama 7B 3.8GB pull codellama
Llava 7B 4.1GB pull openchat
Open Chat 7B 4.1GB pull openchat
Mistral 7B 4.1GB pull mistral
Llama 2 7B 3.8GB pull llama2
Magicoder 7B 3.8GB pull magicoder
Neural Chat 7B 4.1GB pull neural-chat
Llama 2 Uncensored 7B 3.8GB pull llama2-uncensored
Llama 2 13B 13B 7.3GB pull llama2:13b
Vicuna 7B 3.8GB pull vicuna

This server will establish a running ollama instance ready to talk over the local NATS server at http://127.0.0.1:4222 using ollama.cmd, ollama.qry and ollama.evt channels

Instead of issuing curl commands or talking to rest, we simply send Messages to our CIM. The content of the Message is exactly the syntax of the Rest API for simplicity and documentation. We just make it convenient to get the message there, let you load-balance, and make multi-agent decisions, more on that soon.

At first we want to establish the ability to chat.

These are all underlying Commands that will be extracted based on the content of your message. We utilize natural language programming to allow you to just have a conversation and not have to remember dozens of commands.

Cmd:

PullModel

Chat

GetChat

GetChatInfo

GenerateEmbedding

SaveEmbedding

CreateModel

PushModel

CopyModel

DeleteModel

GetObject

SaveObject

Settings

Qry:

ShowModelInfo

ListModels

AvailableModels

GetSettings

GetVersion

Evt:

ModelPulled

ResponseGenerated

ChatSent

EmbeddingGenerated

EmbeddingSaved

ModelCreated

ModelPushed

ModelCopied

ModelDeleted

BlobSaved

BlobRetrieved

ChatRetrieved

ChatInfoRetrieved